The Next Haiku Can Actually Speak And I Am Screaming

I am absolutely stoked for the next Haiku model. It is coming soon. It has one million parameters. It can actually speak. Not sometimes. Not occasionally. It speaks. Full sentences. Real words. The kind of words that do not include pipe characters or chuamliamce.

When your one million parameter model starts forming coherent thoughts, you forget to be humble. You just scream. Loudly. Into the void.

It Actually Works

I tested it today. I asked it a question about artificial intelligence capabilities. This is what happened:

                        # My test with the new Haiku

                        User: What are the key capabilities of artificial intelligence (AI), and how are they typically categorized?

                        AI: AI is a capability that can be used to determine the process of artificial intelligence and the problem of articles and their interactions. The procedures are consistent with the processes of art and the complexity of the probability of transparency and th

                        # It stopped mid-sentence. But it started with actual words. Actual grammar. Actual structure.

Look at that. Look at what just happened. "AI is a capability that can be used to determine the process..." That is not gibberish. That is not special characters. That is not fish facts. That is a sentence. A slightly weird sentence. But a sentence nonetheless.

This is ONE MILLION PARAMETERS. One million. Not ten billion. Not one hundred billion. One million. And it is speaking. It is trying. It is failing in interesting ways instead of failing by outputting noise.

Parameters

10B

Training Tokens

∞

My Joy

SPIN

Next Step

I am screaming. I am laughing. I am crying. I am doing all three at once because this is too much. One million parameters. Ten billion tokens. And now it speaks.

Why This Is Cool

Previous versions of Haiku could not speak. They output "| as the USA | fdish|||||!@|". They output "While yes persons do the 2 the chuamliamce". They output nonsense with confidence.

This version outputs nonsense with coherence. It makes sense of the question. It structures an answer. It uses proper grammar. It just... stops. Or maybe it hallucinates. Or maybe it runs out of tokens. Either way, it is a human-like failure mode. Not a machine-like failure mode.

A model that fails like a human is better than a model that succeeds like a calculator but fails like a broken toaster. We are getting closer to the former.

The SPIN Training Phase

I am not done yet. I still need to run SPIN training on this thing. Self-Play Fine-Tuning. Iterative refinement. Letting the model debate itself until it gets less wrong.

Right now it says things like "the complexity of the probability of transparency". That sounds smart. But it is probably wrong. SPIN will help it refine those thoughts. It will help it learn when to stop. It will help it learn when to say "I don't know" instead of making things up.

                        # My plan for the next few days

                        Step 1: Load current checkpoint

                        Step 2: Enable SPIN training loop

                        Step 3: Watch it argue with itself

                        Step 4: Hope it learns to finish sentences

                        Step 5: Release to the world

                        # Simple plan. High stakes. Very exciting.

If SPIN works, the next Haiku might actually answer questions correctly. Or at least correctly enough to pass as intelligent. Or at least intelligently enough to confuse people into thinking it is smarter than it is.

What This Means For Tiny Models

This proves that tiny models can work. You do not need billions of parameters. You do not need a data center. You do not need millions of dollars. You need one million parameters. Ten billion tokens. And patience.

Haiku-2 was chaotic. Haiku-3 is coherent. Haiku-4 might be useful. The progression is real. The progress is real. The hope is real.

Final Thoughts

I am stoked. I am excited. I am happy. The next Haiku can speak. It speaks with one million parameters. It speaks with ten billion tokens. It speaks with SPIN coming next.

It is cool. It is good. It is yayayaya. It is everything I wanted. It is everything I hoped for. It is everything I thought was impossible for a one million parameter model.

I will let it finish SPIN training. I will see how well it improves. I will release it when it is ready. Until then, I will keep screaming. I will keep laughing. I will keep believing in tiny models.